Pixellated image data sub-sampling

ABSTRACT

A method and a circuit for sub-sampling pixelized image data gathered in overlapping blocks, comprising reading, line by line, from an image memory containing the pixelized image, accumulating as many lines as provided by the sub-sampling ratio in the vertical direction, using as many accumulator groups as there are blocks in the horizontal image direction and as many accumulators per group as provided by the sub-sampling ratio in the horizontal direction, and memorizing the accumulated values in as many result memories as there are accumulator groups, each result memory containing sub-sampled matrixes of a number of blocks corresponding to the number of overlapping blocks in the vertical direction.

The present invention relates to digital image processing and morespecifically to the sub-sampling of digital images. A sub-samplingamounts to approximating a pixelized original image to reduce its size.

Most often, such a sub-sampling is performed regularly in terms ofdimensions, that is, a resulting value is made to correspond to a pixelgroup of given dimensions, the number of pixels in the group beingconstant for the entire image. The set of resulting values forms amatrix that can be related to an image of reduced dimensions. The datacontained by this matrix generally correspond to an average valuedepending on the original image data (for example, grey level or levelin a given color, etc.).

The present invention more specifically applies to the sub-sampling ofdigital images divided into overlapping blocks. In the sense of thepresent invention, it is considered that blocks overlap if the number ofpixels separating the origins of the two blocks is smaller than thesub-sampling ratio in the vertical direction and/or in the horizontaldirection. The sub-sampling ratio corresponds to the number of pixelstaken to calculate a single resulting value.

An example of application of the present invention relates to thefractal coding of digital images and more specifically to the step ofsub-sampling the so-called domain blocks of such a fractal coding.

Fractal image coding comprises searching, in a digital image, portionsof this image that can be considered as being identical after having, ifnecessary, undergone simple transformations (symmetry, rotation) calledisometries.

FIG. 1 shows an image I to be submitted to a fractal coding andillustrates the division of this image.

A search window SW in which it is desired to determine if image portionsmay relate to a reference block RB of the image is first defined inimage I. The search window corresponds, at most, to the size of image I.

To obtain image blocks, called domain blocks, to be compared with thereference image, image blocks of larger dimensions B1, B2, etc. aresub-sampled to be brought to the size desired for the domain blocks. Thesize of the domain blocks corresponds to the size of the searchedreference block, generally called “range”. For example, the referenceblock and the domain blocks correspond to 8*8 matrixes while blocks Btaken in the search window are blocks of 32*32 pixels.

FIG. 2 illustrates the sub-sampling performed from blocks B1, Bi, B9 toobtain reduced domain blocks DB1, DBi, DB9.

In a fractal coding method, overlapping original blocks are selected inthe search window, which is indispensable to determine the possibleidentity of the domain blocks, with respect to the reference blocks ortheir isometries, across the entire search window.

The comparison of the domain blocks and of the reference blocks isperformed either by pixel-to-pixel grey level difference or by addingthe differences of the squares between the respective values of thedomain and reference blocks.

Digital image fractal coding is an image compression technique used toreduce the volume of image transmissions. Rather than sending all theblocks of an image, for similar blocks, the reference block is only sentonce, followed by the number of this block and of its possible isometryto define the similar domain blocks.

The fractal image coding or compression technique is described, forexample, in work “Fractal image compression: Theory and application todigital images” by Yuval Fisher, published by Springer Verlag, New-York,1995. Another example of a fractal image compression algorithm isdescribed in article “Design of an ASIC architecture for high speedfractal image compression” by Ancarani De Gloria and Olivieri Stazzone,published in IEEE “International ASIC conference”, September 1996.

Those skilled in the art can also refer to French patent application2,775,812.

The number of operations and of memory accesses to be performed tosub-sample the blocks of the search window to obtain the domain blocksis very high. Indeed, a sequential reading, pixel by pixel, from amemory containing the pixel values of the search window, isconventionally performed to isolate the different blocks and calculatetheir respective sub-sampled values.

The sampling includes calculating the average of the individual valuesof the pixels of the sub-blocks to be sub-sampled. In other words, therespective values of the pixels of each sub-block (for example, thesixteen values of the sub-blocks of 4*4 pixels, referring to thepreceding example) are added and the result is divided by the number ofpixels of the sub-blocks to obtain the average value as the sub-samplevalue.

It has already been provided, to reduce the number of memory accesses,to keep the intermediary sums to only perform four extractions of pixelvalues for each new sampled value.

Sequentially extracting the sixteen values of each sub-block of theimage memory requires, even while keeping the intermediary sums,performing for each pixel of the domain blocks (thus, sub-sampled)sixteen memory accesses, sixteen additions, and one division.

The present invention aims at providing a novel method and system forsub-sampling overlapping digital images which reduce the number ofmemory accesses and of calculations necessary to obtain the sub-sampledimages.

The present invention also aims at providing a solution which can adaptto different sub-sampling ratios and image dimensions.

The present invention also aims, without excluding a softwareimplementation, at providing a solution enabling a particularly simplehardware implementation.

To achieve these and other objects, the present invention provides amethod for sub-sampling pixelized image data gathered in overlappingblocks, including the steps of:

reading, line by line, from an image memory containing the pixelizedimage;

accumulating as many lines as provided by the sub-sampling ratio in thevertical direction, using as many accumulator groups as there are blocksin the horizontal image direction and as many accumulators per group asprovided by the sub-sampling ratio in the horizontal direction; and

memorizing the accumulated values in as many result memories as thereare accumulator groups, each result memory containing sub-sampledmatrixes of a number of blocks corresponding to the number ofoverlapping blocks in the vertical direction.

According to an embodiment of the present invention, the memorization isperformed in an interlaced fashion.

According to an embodiment of the present invention, the accumulatedvalues are divided by the product of the sub-sampling ratios in bothdirections, to obtain average values to be memorized as sub-samples.

According to an embodiment of the present invention, the division of anaccumulated value of several lines of the image memory to obtain anaverage value is obtained by only taking into account a number of mostsignificant bits, smaller than the number of bits of the result value.

According to an embodiment of the present invention, the lines of theimage memory are read successively from the first one for a number oflines corresponding to the sub-sampling ratio in the vertical direction,after which the first following line and a previously-used line arealternately read.

The present invention also provides a circuit for sub-sampling pixelizedimage data distributed in overlapping blocks, including:

a number of adders corresponding to the number of result block pixels ina first direction, multiplied by the number of blocks in a seconddirection;

a number of accumulators identical to the number of adders; and

a number of result memories of the sub-sampled values corresponding tothe number of blocks in the first direction.

According to an embodiment of the present invention, the accumulatorsare controllable for addition or subtraction of a current value to thepreviously-accumulated result.

According to an embodiment of the present invention, the number ofinputs of each adder corresponds to the sub-sampling ratio in the firstdirection.

According to an embodiment of the present invention, said resultmemories include a number of lines corresponding to the number of blocksin the second direction, multiplied by the number of pixels of theresult blocks in the second direction.

According to an embodiment of the present invention, the number of bitsof a result value stored in one of said result memories is smaller thanthe number of bits of the values of the pixelized image data, thedifference between the two numbers of bits defining the division ratiofor obtaining the average value of the pixels of each sub-sampled group.

The foregoing objects, features and advantages of the present inventionwill be discussed in detail in the following non-limiting description ofspecific embodiments in connection with the accompanying drawings, inwhich:

FIGS. 1 and 2, previously described, illustrate an example of an imageand of a sub-sampling to which the present invention applies; and

FIG. 3 shows an embodiment of an image block sub-sampling circuitaccording to the present invention.

Same elements have been designated with same references in the differentdrawings. For clarity, only those circuit elements and those methodsteps which are necessary to the understanding of the present inventionhave been shown in the drawings and will be described hereafter. Inparticular, the image processing upstream and downstream of theimplementation of the method of the present invention has not beenillustrated and are no object of the present invention. Further, thecircuits for addressing the memories used by the present invention havenot been detailed, being within the abilities of those skilled in theart based on the functional indications of the present invention.

A feature of the present invention is to organize the sub-sampling ofthe overlapping blocks of an image by simultaneously processing allvalues of the pixels of the search window. Thus, according to thepresent invention, a memory containing the values of the pixels of thesearch window, in a matrix arrangement similar to that of the image, isaddressed simultaneously for all its columns, to simultaneously processan entire line. Accordingly, the scanning of the memory according to thepresent invention is performed line by line.

Another feature of the present invention is to process, in parallel, thesub-sampling of the overlapping blocks in the line direction. In otherwords, as many series of arrangement operators as there are overlappingblocks in the line direction are provided.

The present invention will be described hereafter in relation with anexample of application to the sub-sampling of blocks of a search windowto obtain domain blocks in a fractal coding of an image. It shouldhowever be noted that the present invention more generally applies toany sub-sampling of image blocks overlapping in at least one direction.

To simplify, reference will be made to lines and columns respectivelycorresponding to the horizontal and vertical image directions. It shouldhowever be noted that these notions of direction are arbitrary and maybe inverted without changing the principles of the present invention.

FIG. 3 shows an example of architecture of a circuit for sub-sampling animage memory according to the present invention. In this example, thecase of a search window SW such as illustrated in FIG. 1, that is,comprised of 34 columns and 34 lines, is considered. The search windowis comprised of 9 blocks of 32*32 pixels which must be sub-sampled in 9domain blocks of 8*8 pixels. In practice, a larger search window, forexample, of 64 by 64 pixels, is used. To simplify, the present inventionwill be described in relation with a search window of 34 by 34 pixels.It however applies whatever the number of pixels of the search windowand of the blocks.

The sub-sampling correspond, as previously, to taking the average ofgroups of 16 pixels (4*4) of each block to form a pixel of the resultingdomain block.

The case of an image in levels of grey will be considered hereafter.Obtaining the average thus amounts to adding the respective levels ofgrey of the pixels of each group and of dividing the obtained number bythe number of pixels. As an alternative, the division may be omitted ifthe multiplication factor of the number of pixels of each group is takeninto account in the reference block (RB, FIG. 1).

A sub-sampling circuit according to the present invention includes asmany inputs as the search window to be processed includes columns, thatis, as many inputs as there are memory words per line to be processed.

According to the present invention, blocks B1 to B9 are processed 3 by 3to minimize the number of operations to be executed. The 3 by 3processing corresponds to the number of block overlappings present inthe search window in each direction.

For each series of blocks, as many adders as there are average values ineach line of the domain block are provided. Referring to the precedingexample, a line of adders S11 to S18 is provided for the first group ofblocks B1, B4, and B7 to be processed. Each adder S11 to S18 receives asan input four values sampled from memory M1. Each adder S11 to S18 isassociated at its output to an accumulator A11 to A18 intended to addthe successive values of the lines of each pixel group forming anaverage value of the concerned domain block.

According to the present invention, the accumulators are controllable toincrement or decrement the accumulated value with the current value. InFIG. 3, accumulators Aij (i representing the line to which theaccumulator or the adder belongs, and j representing the horizontal rankof the average value of the concerned domains) have been symbolized bytwo-input blocks, a first input being looped back on the output while asecond input is connected to the output of the corresponding adder Sij.The accumulators are controllable not only in a configuration ofaddition or subtraction of the current value, but also in a resetconfiguration, if necessary.

Referring to the example of the processing of the search window of FIG.1, the first line of adders S11 to S18 is connected to the respectivecolumns 1 to 32 of memory M1. The second line of adders S21 to S28 isconnected to the respective columns 2 to 33 of memory M1. The third lineof adders S31 to S38 is connected to the columns 3 to 34 of memory M1.

According to a preferred embodiment of the present invention, thedivision is performed by only taking a given number of most significantbits of the result obtained in the accumulators. The retained number ofbits depends on the desired division factor. In the case of a binaryword over 12 bits and of a division by 16, only the 8 most significantbits of the obtained result are taken. This amounts to performing adivision by sixteen, rounded to the smallest integer. Such a divisionmode is particularly simple and has the remarkable advantages ofrequiring no calculation circuit and no cycle time to perform thecalculation. The price to pay is an approximation by a lower value. Suchan approximation is however not prejudicial since the result in allcases is a sub-sampling. In FIG. 3, the selection of the 8 mostsignificant bits has been illustrated by cuttings of the connections ofthe accumulators to the memories. The selection amounts to only taking 8wires out of the 12 connection wires.

According to the present invention, the results of the respectiveaccumulators are stored in three result memories MR1, MR2, MR3 (or threeareas of a same memory) corresponding to the obtained domain blocksgathered in columns. The storage in the result memories is performedeach time an accumulator contains a complete value. This synchronizationis performed by means of a control circuit which will be describedfunctionally hereafter.

The result obtained in this memory preferably corresponds to the domainblocks stored in interlaced fashion, the reading from the resultmemories being then adequately controlled to properly restore the domainblocks. As an alternative, the reading will be performed line by linebut the storage of the results of the accumulators in the differentmemories is controlled as appropriate. The embodiment corresponding toan interlaced storage simplifies the recording since it is enough toincrement the addresses of the result memories of a unit for each newrecording.

The addressing of memory M1 is performed, preferably, line by line(addresses AD1, AD2, etc.).

According to the present invention, the control of the sub-samplingcircuit is performed as follows.

The first line of memory M1 is first read and the respective sumsprovided by adders Sij are added in the respective accumulators Aijpreviously reset to zero.

The second, third, and fourth lines of memory M1 are then successivelyread and the sums by groups of four columns by means of circuits Sij areadded to the preceding values of the accumulators Aij. At the end of thefourth line, the first value lines of domain blocks D1, D2, and D3 areobtained in the respective accumulators.

These values are then stored in the first lines of result memories MR1,MR2, and MR3, respectively.

The average values which are then calculated correspond to the firstlines of domain blocks D4, D5, and D6. For this purpose, the first lineof memory M1 is first reread, and the respective sums by groups of fourcolumns of the levels of grey are subtracted from the values previouslyaccumulated in accumulators Aij. They then contain the accumulation oflines 2, 3, and 4. The fifth line of memory M1 is then read, and addedto the content of accumulators Aij by groups of four columns. The resultof the accumulators (lines 2, 3, 4, and 5) is then unloaded towards therespective second lines of memories MR1, MR2, and MR3, possiblyundergoing the division.

The next average value calculation step consists of subtracting thesecond line of memory M1 and of adding the sixth line. The obtainedresult then corresponds to the first line of average values of domainblocks D7, D8, and D9.

To calculate the average values of the second line of domain blocks D1,D2, and D3, two solutions are possible. A first solution comprisesresetting the accumulators and of successively reading positivelyaccumulated lines 5, 6, 7, and 8 to form the searched average values. Asecond solution comprises rereading lines 3 and 4 to subtract them fromthe precedingly accumulated values, then reading lines 7 and 8 andadding them to the accumulator results. Whatever the solution, this steprequires four calculation and memory access cycles.

The second solution however is a preferred embodiment since it enablessimplification of the sequencing algorithm of the sub-sampling circuitcontrol. Indeed, this solution respects the succession of subtractionand addition steps in the accumulators. It is enough to start withsubtracting line 3 and adding line 7, then subtracting line 4 and addingline 8.

The above-described sequencing carries on for the rest of memory M1, toobtain the last line of domain blocks D7, D8, and D9 corresponding tothe last reading of line 34 of memory M1.

The total number of calculation cycles necessary to sub-sample theentire memory in the above example is 64 cycles (corresponding to thenumber of pixels of a sub-sampled block). Cycle means the time periodnecessary to perform an addressing and a reading of the datum from thememory as well as the corresponding addition. This addition is howeverperformed with no time consumption since it can be performed by means ofsimple logic gates. The cycle time more generally corresponds to thememory access time plus the addition and accumulation time.

The number of 64 cycles indicated hereabove is to be compared with anumber of 16×9×64 cycles necessary in case of an addressing of a memoryby sub-groups of 16 pixels (4×4) to calculate the successive averagevalues in the conventional case.

An advantage of the present invention is that it enables considerablyreducing the number of memory accesses and the time required tosub-sample a search window.

Another advantage of the present invention is that it is particularlysimple to implement by hardware means with simple circuits.

Of course, the present invention is likely to have various alterations,modifications, and improvements which will readily occur to thoseskilled in the art. In particular, although the present invention hasbeen described hereabove in relation with an example of application to afractal coding search window, it more generally applies whatever thesize of the search window and the number of overlapping image blocks tobe sub-sampled. Generally, considering an image or search window of n*mpixels defining the size (number of lines and number of columns) ofmemory M1, and assuming blocks to be sub-sampled of p*q pixels with p<nand q<m, the implementation of the present invention can be described asfollows. It must first be assumed that the blocks of p*q pixels overlapor cover one another, that is, number k of blocks in the verticalalignment is greater than n/p and number 1 of blocks in the horizontalalignment is greater than m/q. Number k defines the number of interlacedblocks in each result memory MR. Number 1 defines the number of resultmemories, and thus of adder lines and of accumulator lines. The numberof columns of the result memories, which corresponds to the number ofadders and of accumulators per line, is a function of the desiredsub-sampling ratio in the horizontal direction, as well as of the numberof inputs of each adder. The total number of adders and accumulatorsthus corresponds to the number of pixels of the result blocks in thehorizontal direction multiplied by the number of blocks in the verticaldirection. The desired sub-sampling ratio in the vertical direction iscontrolled by the circuit controlling the unloading of the accumulatorsinto the result memories. The number of lines of each result memorycorresponds to the number of blocks in the vertical direction,multiplied by the dimension (p) of the block in the vertical directionand divided by the sub-sampling ratio in this vertical direction, andthus to the number of blocks in the vertical direction multiplied by thenumber of pixels of the result blocs in the vertical direction.

The practical realization of the sub-sampling circuit and of itsselectors and control circuits is within the abilities of those skilledin the art based on the functional indications given hereabove. Further,the implementation of the present invention for other applications thanthose described as an example is also within the abilities of thoseskilled in the art based on the functional and generalizationindications given hereabove.

Finally, the present invention applies whatever the data to be processedcontained in the image memory. It may be levels of grey in the case ofblack and white images or color levels, or data relative to the contrastof the different pixels. The number of bits of each memory wordrepresenting a pixel depends on the application and is not critical inthe sense of the present invention. Further, the last lines of the imagememory may undergo a specific processing for the case where the sizes ofthe blocks to be sub-sampled result in a number of lines that does notexactly fit into the image memory. The same may be done for the lastcolumns.

1. A method for sub-sampling pixelized image data gathered inoverlapping blocks, comprising: reading, line by line, from an imagememory containing the pixelized image data; accumulating as many linesas provided by a sub-sampling ratio in a vertical direction, using asmany groups of accumulators as there are blocks in a horizontal imagedirection and as many accumulators per group as provided by thesub-sampling ratio in the horizontal direction; and memorizing theaccumulated values in as many result memories as there are accumulatorgroups, each result memory containing sub-sampled matrixes of a numberof blocks corresponding to a number of overlapping blocks in thevertical direction.
 2. The method of claim 1, wherein the memorizationis performed in an interlaced fashion.
 3. The method of claim 1, furthercomprising dividing the accumulated values by a product of thesub-sampling ratios in both directions, to obtain average values to bememorized as sub-samples.
 4. The method of claim 3, wherein the divisionof an accumulated value of several lines of the image memory to obtainan average value is obtained by only taking into account a number ofmost significant bits, smaller than a number of bits of a result value.5. The method of claim 1, wherein the lines of the image memory are readsuccessively from a first one for a number of lines corresponding to thesub-sampling ratio in the vertical direction, after which a firstfollowing line and a previously-used line are alternately read.
 6. Acircuit for sub-sampling pixelized image data distributed in overlappingblocks, comprising: a number of adders corresponding to a number ofresult block pixels in a first direction, multiplied by a number ofblocks in a second direction; a number of accumulators identical to thenumber of adders; and a number of result memories of sub-sampled valuescorresponding to the number of blocks in the first direction.
 7. Thecircuit of claim 6, wherein the accumulators are controllable foraddition or subtraction of a current value to a previously-accumulatedresult.
 8. The circuit of claim 6, wherein a number of inputs of eachadder corresponds to a sub-sampling ratio in the first direction.
 9. Thecircuit of claim 6, wherein said result memories include a number oflines corresponding to the number of blocks in the second direction,multiplied by the number of pixels of the result blocks in the firstdirection.
 10. The circuit of claim 6, wherein a number of bits of aresult value stored in one of said result memories is smaller than anumber of bits of the values of the pixelized image data, a differencebetween the two numbers of bits defining a division ratio for obtainingan average value of pixels of each sub-sampled group.
 11. An apparatus,comprising: a first memory to store pixelized image data arranged inoverlapping blocks; a plurality of adders coupled to the memory andhaving input terminals to receive first values from the first memorythat correspond to the stored pixelized image data and to output secondvalues; a plurality of accumulators respectively coupled to an outputterminal of the adders to obtain third values based at least in part onsecond values output from their respective adders; and at least onesecond memory coupled to output terminals of the accumulators to storethe third values obtained by the accumulators as domain blocks having areduced size relative to the overlapping blocks.
 12. The apparatus ofclaim 11 wherein the plurality of adders have a number that correspondsto a number of result block pixels, of the stored pixelized image data,in a first direction, multiplied by a number of blocks of the pixelizedimage data in a second direction.
 13. The apparatus of claim 12, furthercomprising a plurality of additional second memories, the secondmemories corresponding in number to a number of blocks in the firstdirection.
 14. The apparatus of claim 11 wherein a number of theaccumulators is identical to a number of the adders.
 15. The apparatusof claim 11 wherein only certain output terminals of the accumulatorsprovide the third values to the second memory.
 16. The apparatus ofclaim 11 wherein the accumulators are coupled to receive, at one oftheir input terminals, the third values from their output terminals. 17.The apparatus of claim 11 wherein the accumulators are controllable toadd or subtract a current value from a previously obtained value. 18.The apparatus of claim 12 wherein a number of input terminals of eachadder to receive the first values from the memory corresponds to asub-sampling ratio in the first direction.
 19. The apparatus of claim 12wherein the second memory includes a number of lines corresponding tothe number of blocks in the second direction, multiplied by a number ofresult block pixels.
 20. The apparatus of claim 11 wherein the thirdvalues stored in the second memory include sub-sampled matrices of anumber of blocks corresponding to a number of blocks of the pixelizedimage data in a certain direction.